NON-NILPOTENT GRAPH OF A GROUP 



ALIREZA ABDOLLAHI AND MOHAMMAD ZARRIN 

Abstract. We associate a graph J\[q with a group G (called the non-nilpotent 
graph of G) as follows: take G as the vertex set and two vertices are adjacent 
if they generate a non-nilpotent subgroup. In this paper we study the graph 
theoretical properties of Mg an d its induced subgraph on G\nil(G), where 
nil(G) = {x £ G | (x, y) is nilpotent for all y 6 G}. For any finite group G, 
we prove that Mg na s either |Z*(G)| or |Z*(G)| + 1 connected components, 
where Z*(G) is the hypercenter of G. We give a new characterization for finite 
nilpotent groups in terms of the non-nilpotent graph. In fact we prove that a 
finite group G is nilpotent if and only if the set of vertex degrees of A/q has at 
most two elements. 



1. Introduction and results 

Study of algebraic structures by graphs associated with them gives rise to many 
recent and interesting results (see e.g., pQ, [3]). Here we associate a simple graph 
with a group by the property of being nilpotent. P. Erdos has introduced a graph 
associated with a group by the property of commutativity: it is a graph whose 
vertex set is the set of elements of the group and two vertices are adjacent whenever 
they do not commute (see [21] and its review by Erdos in Mathematical Review 
MR0419283 (54#7306)). Let G be a group. We associate a simple graph Af G with 
G (called the non-nilpotent graph of G) as follows: take G as the vertex set and join 
two vertices by an edge if they do not generate a nilpotent subgroup. Our aim is to 
study a group G by the information stored in its non-nilpotent graph and to study 
how the graph theoretical properties of Ng effect on the group ones of G. Note that 
if G is weakly nilpotent (i.e., every two generated subgroup of G is nilpotent), No 
has no edge. It follows that the non-nilpotent graphs of weakly nilpotent groups 
with the same cardinality are isomorphic. So we must be interested in non weakly 
nilpotent groups. On the other hand, as there are vertices in Afc which are isolated 
(e.g., elements in the hypercenter of G), it is wise to study the subgraph of Ng 
induced by G\nil(G), where 

nil(G) = {x G G | (x,y) is nilpotent for all y £ G}. 

It is not known whether the subset nil(G) is a subgroup of G, but in many important 
cases it is a subgroup. In particular, nil(G) is equal to the hypercenter Z*(G) of 
G whenever G satisfies the maximal condition on its subgroups or G is a finitely 
generated solvable group (see Proposition ^. 1[ below). 

The set of adjacent vertices to a vertex v in a graph T is denoted by Nr(v). For 
an element x of a group G, the set Nj^(x) U {x} will be denoted by nilcix), where 
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Mg is the complement of the non-nilpotent graph of G. In fact 

nilQ(x) = {g £ G (x,g) is nilpotent}. 

We call nilcix) the nilpotentizer of x in G. In Section 2 we show some results on 
the nilpotcntizers. 

In Section 3 we give some properties of n-groups, where by an n-group we mean 
a group G in which nilcix) is a subgroup for every x € G. We also study some 
possible relations between simple n-group and some classes of groups (see Theorem 
EH below). 

A set C of vertices of a graph T whose induced subgraph is a complete sub- 
graph is called a clique and the maximum size (if it exists) of a clique in a graph 
is called the clique number of the graph and it is denoted by t^(r). In Section 4 
we state some previously proven results about groups whose non-nilpotent graphs 
have finite clique number. We prove some results on the girth and diameter of 
the non-nilpotent graph also we show that there is no finite non-nilpotent group 
G such that D^g — or 91g/jv for some proper subgroup H or quotient G/N of G. 

In Section 5 we shortly prove the connectedness of O^g for finite non-nilpotent 
groups G. Also, in this case, we estimate the diameter of D^g to be at most 6 while 
we conjecture that the best bound for the diameter must be 2. 

A planar graph is a graph that can be embedded in the plane so that no two 
edges intersect geometrically except at a vertex which both are incident. In Section 
6 we show that for a non-nilpotent finite group G, 9!g is planer if and only if G = £3. 

The degree of a vertex v in a graph T is the number of vertices which are adjacent 
to v in r. A graph T is said to have m kind vertex degrees if the set of vertex degrees 
of r is of size m. So by definition, a graph is regular if and only if it has 1 kind 
vertex degree. In Section 7 we prove that a finite group G is nilpotent if and only 
if A/"g has at most two kind vertex degrees. This shows that there is no finite 
non-nilpotent group G such that 9Ig is regular. 

2. Some properties of nilpotentizer 

Let G be a group and x be an element of G. The set of vertices of the complement 
graph Mg of the non-nilpotent graph G which are adjacent to x together with x 
itself consist a set which we call it the nilpotentizer of x in G. It will be denoted 
by nilcix) . Indeed 

nilcix) = {g £ G | (x, g) is nilpotent} 
For a nonempty subset S of G, we define the nilpotentizer of S in G as 

nilc{S) = P| nil G (x). 

xes 

We call nilc{G) the nilpotentizer of G, and it will be denoted by nil(G). Thus 

nil(G) — {x £ G I (x, y) is nilpotent for all y £ G}. 

Besides nilpotentizers of elements in a group which are not necessarily subgroups 
even in finite solvable groups (see Lemma 13.31 below) , there are vast classes of 



NON-NILPOTENT GRAPH OF A GROUP 



3 



groups, in which nilpotcntizcrs of groups are subgroups. In Section 3, we will study 
n-groups, that is, groups in which the nilpotentizer of each element is a subgroup. 

Proposition 2.1. Let G be any group. Then 

(1) Z*(G) C nil(G) C R(G), where R(G) is the set of right Engel elements of 
G. 

(2) // G satisfies the maximal condition on its subgroups or G is finitely gen- 
erated solvable group then Z*(G) = nil(G) = R(G). 

Proof. (1) It is straightforward. 

(2) It follows from [23] Theorem 12.3.7] and the main result of [7]. □ 

Lemma 2.2. Let G be any group, N a normal subgroup of G and x,g G G. Then 

(1) nil(G) C nil G (x). 

(2) mloME c nila(xN). {obviously here Sil^ME ■- {yN \ y € nil G (x)N}). 

(3) nila_{xK) = m 'gfr) f where K is a normal subgroup of G with K < Z*(G). 

(4) G is an n-group if and only if % is an n-group for some normal subgroup 
KofGwithK<Z*{G). 

(5) // (x) = (y) for some y £ G, then nilc{x) = nilc(y)- 

(6) nilQ{x) 9 = nilcix 9 ). 

Proof. The proof is straightforward. □ 

Lemma 2.3. Let G be a finite group and a be an element of G. Then mZc(a) is 
divisible by \a\. 

Proof. It is easy to see that nilcia) is the union of maximal nilpotent subgroups 
of G containing a. On the other hand, each of these subgroups is a union of cosets 
of (a). It follows that the order of a divides \nilG(a)\. □ 

3. Groups in which nilpotentizers of elements are subgroups 

We call a group G is an n-group if nilc(x) is a subgroup of G for every x G G. In 
this section we study n-groups. Following 8 a finite group G is called an sn-group 
whenever 

Sg(x) = {g € G | (x) is a subnormal subgroup of (x,g)} 

is a subgroup of G for all x G G and also G is called an sn(p) -group (for some prime 
p) if Sg{x) < G for all p-elements x £ G. For example, the dihedral group D 2 m is 
not only an n-group but also an sn-group. There is another related class of groups, 
called i^-groups: A group G is called an iJ-group if 

Eq{x) — {g G G [g, n x] — 1 for some positive integer n} 

is a subgroup of G for every x 6 G; and for a prime p, G is called an _E p -group if 
Eg{x) < G for all p-elemcnts x G G. These class of groups was defined and studied 
by Peng [22]. Finite sn-groups has been studied by Casolo [8] as well as Heineken 
[15] . Note that nilc{x) C Sg(x) C Eg{x) for any group G and x G G. The relation 
between these two latter classes of groups is established in the following result. 

Theorem 3.1. ([S], Theorem 1.11) Let G be a finite group. The following are 
equivalent: 

(1) G is an sn(p)-group for every p dividing \G\. 
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(2) G is an sn-group; 

(3) G is an -E-group; 

(4) G is an _E p -group for every p dividing \G\. 

The non-abelian finite simple sn-groups are classified. 

Theorem 3.2. ([8], Theorem 2.1) A non-abelian finite simple group is an sn-group 
if and only if it is one of the following groups: 

PSL(2,2 n ), n > 2; Sz(2 2m+1 ), m > 1 

Besides uUq{x) C Sg{%) for any element a; of a group G, there are sn-groups 
which are not n-groups. 

Lemma 3.3. The symmetric group S4 of degree 4 is an sn-group which is not an 
n-group. 

Proof. We note that (e.g., by GAP [13]) that 

m/ S4 ((12)(34)) = {(), (34), (13), (23), (24), (14), (12), (12)(34), (13)(24), 
(14)(23), (1324), (1432), (1234), (1243), (1342), (1423)}. 
Thus 5*4 is not an n-group and it follows from Lemma 1.15 of [5] that S4 is an 
sn-group. □ 

For a prime number p, we call a group G an n p -group whenever nildx) is a 
subgroup of G for all p-elements x of G. We denote by w(x) the set of all primes 
dividing of \x\ for any element 1 in a finite group. 

Lemma 3.4. A finite group G is an n-group if and only if G is an n p -group for all 
primes p dividing \G\. 

Proof. Suppose that G is an n p -group for all prime numbers p dividing \G\. Let x 
be a nontrivial element of G and k be the number of prime divisors of \x\. We argue 
by induction on k to prove that nilc{x) is a subgroup of G. If k = 1, then the proof 
follows from the hypothesis. Thus we may assume by induction that k > 1 and for 
every nontrivial element a of G with |7r(a)| < k, nila(a) is a subgroup of G. Let 
y, z G nilcix). We have to prove that K = (yz, x) is nilpotent. To prove that K is 
nilpotent, it is enough to find a positive integer m such that [zi, . . . , z m ] = 1 for all 
Zi G {yz, x}. Since |7r(x)| > 2, there are nontrivial commuting elements a and b of 
G such that x = ab, a is an p-element and b is an p'-element for some prime number 
p. Now by induction hypothesis nilc(b) < G. Since both a and b are powers of 
x, we have y, z G nilc(a) (~l nilQ(b). Hence, by hypothesis, yz G nilc(a) D nila(b). 
Since a is a p- element and [yz,a] belongs to the nilpotent group {yz,a), it follows 
that [y.z,a] is also a p-element. Also [yz,b] is a p'-element. Now since (yz,a) is 
nilpotent, 

yz £ nil G ([yz,a}); (1) 

and since yz,a G nilalb), we have [yz, a] G nilc(b) and equivalcntly 

benil G ([yz,a}). (2) 

It follows from (1) and (2) that [yz,b] G nilc{[yz, a]) and so ([yz, b], [yz, a]) is 
nilpotent. Now since gcd(| [yz, a]\, \ [yz, b] |) = 1, [yz,a] commutes with [yz,b\. Also 
it follows from (2) that b commutes with [yz, a]. Hence 

[yz,x] = [yz,ab] = [yz, b][yz, a] b = [yz,a][yz,b]. 
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Now let yz = cd where c and d are two commuting elements of G such that c 
is an p-element and d is an p'-element. Note that c, d £ nil G (a) f~l nil G (b), since 
they are powers of yz. By a similar argument we may prove that [yz, a] = [c,a], 
[yz, b] — [d, b] and by continuing in this manner we have that 

[zi, ...,z s ] = [ti, . . .,t„][yi, ...,y s ] 

where Zi £ {yz,x}, ti £ {a,c} and yi £ {b, d}. Let m be the maximum of the 
nilpotency classes of (a, c) and (b,d). Thus 

[ti, ...,t s ] = [z/i, . . . ,y s ] = 1 for all s > m. 

Therefore (yz,x) is nilpotent of class at most m. This completes the proof of the 
"if" part. 

The converse is obvious. □ 

Lemma 3.5. Let p be a prime number. If the Sylow p-subgroups of a finite group 
G are abelian, then G is an n p -group. In particular, nil G (x) — C G {x) for all 
p-elements x £ G. 

Proof. Let y £ nil G (x) and y — y\y 2 where y\ is an p-element, y 2 is an p'-element 
and [y\, y 2 ] = 1. Then (x, y) = (x, y\,y 2 ) is nilpotent. Thus [y 2l x] = 1 and as (x, y\) 
is a p-group, it follows from the hypothesis that [a;,yi] = 1. Therefore [x,y] = 1 
which means that y £ Ca(x). This completes the proof, as Cq{x) C nilcix). □ 

A group G is an AC-group if Cc(g) is abelian for all g £ G \ Z(G). We can now 
see that some classes of AC-groups are n-groups. 

Lemma 3.6. If G is a centerless (not necessarily finite) AC-group, then G is an 
n-group. In particular, niZg(o) = Cq{cl) for all nontrivial elements a of G. 

Proof. Suppose that a £ G \ {1} and b is an element of G such that (a,b) is 
nilpotent. Then Z((a,b)) ^ 1 and also Z((a,b)) < Cc(a) n Cg(6). Since G is an 
AC-group, C G {a) n C G (b) = Z(G) or ab = ba. If C G (a) n C G (b) = Z(G), we have 
1 ^ Z((a, b)) < C G (a) n C G {b) = Z(G) = 1, a contradiction. Thus ab = ba, that is 
nil G (a) — C G (a). □ 

Now we study some famous families of finite non-abelian simple groups for to be 
a n-group. We show that every simple sn-group is an n-group. 

Lemma 3.7. Let G be a group and H a nilpotent subgroup of G in which C G (x) < 
H for every x £ H\{1}. Then nil G (x) = H for every x £ H\{1}. 

Proof. Suppose y £ nil G (x). Then (y,x) is a nilpotent subgroup. Thus there exists 
a nontrivial element z £ Z((y,x)). Hence z £ C G (x) < H and so z £ H. It follows 
that y £ C G (z) < H. Hence nil G {x) < H. Now since H is a nilpotent subgroup, 
nil G (x) = H . This completes the proof. □ 

Theorem 3.8. Every simple sn-group is an n-group. 

Proof. By Theorem [3J2 it is enough to show that G x = PSL(2, 2") and G 2 = 
Sz(2 2m+1 ) are n-groups for n > 2 and m > 1. The group G\ is a centerless AC- 
group (see e.g., [3l Proposition 3.21, case 7]). Thus by Lemma l3~6l G\ is an n-group. 
The Suzuki group G 2 contains subgroups F, A, B and C such that the set 

V = {A X \{1}, B X \{1}, C X \{1], F X \{1} | x £ G 2 } 
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is a partition for G 2 \{1} and Cg 2 (6) <MU{1} for all b G M and for every M G 7>. 
Now let a 6 G2 \ {1}. Since "P is a partition of G2\{1}, a G M for some M <E P. 
Now Lemma [3.71 implies that nilG 2 ( a ) = M U {1}. Hence Gi is an n-group. This 
completes the proof. □ 

Lemma 3.9. Let q be a prime power number such that q 2 ^ 1 mod 16. Then 
G = PSL(2,<7) is an n-group. In particular, nilc{a) = Cg{o) for all nontrivial 
elements a G G. 

Proof. If q G {3, 5} or q is a power of 2, G is an AC-group and so we are done 
by Lemma 13.61 Thus we may assume q > 5 and gcd(g, 2) = 1. Then it follows 
from [TSl Satz 8.10, Chapter II] that all Sylow subgroups of G are abelian (note 
that the hypothesis q 2 ^ 1 mod 16 is used to prove that the Sylow 2-subgroups 
are elementary abelian) . Now the proof completes by Lemmas 13.51 and 13.41 □ 

Note that by Lemma 2.3 of [5], groups stated in Lemma [331 are sn(2)-groups but 
by Theorem 2.1 of [5] they are not sn-groups. 

Lemma 3.10. Let q be a prime power number such that q 2 = 1 mod 16. Then 
G = PSL(2, q) is not an n-group. 

Proof. By [18] 8.18 Satz, chapter II], G has a subgroup isomorphic to S4. Now 
Lemma 13.31 completes the proof, since the class of n-groups is clearly closed under 
taking subgroups. □ 

4. Groups whose non-nilpotent graphs has finite clique number 

Before starting to show the results, we recall some concepts for a simple graph 

A. A path P in A is a sequence vq — v± — . . . — Vk whose terms are vertices of A such 
that for any i G {1, . . . , fc}, and Vi are adjacent. In this case P is called a path 
between vq and The number k is called the length of P. If v and w are vertices 
in A, then d(v, w) denotes the length of the shortest path between v and w and we 
call d(v, w) the distance between v and w in A. We say that A is connected if there 
is a path between each pair of the vertices of A. If A is connected graph, then The 
largest distance between all pairs of the vertices of A is called the diameter of A, 
and it is denoted by dim(A) and also if A is disconnected graph, then wc define 
the diameter of A as following: 

dim (A) = max{dim(Ai)|A; is a connected component of A}. 

One of our motivations for associating with a group such kind of graph is a problem 
posed by P. Erdos: For a group G, consider a graph G whose vertex set is G and 
join two distinct elements if they do not commute. Then he asked: Is there a finite 
bound for the cardinalities of cliques in G , if G has no infinite clique? 

B. H. Neumann [21], answered positively Erdos problem by proving that such groups 
are exactly the center-by-finite groups and the index of the center can be considered 
as the requested bound in the problem. Wiegold and Lennox [20] proved that if 
G is a finitely generated solvable group in which every infinite subset contains two 
distinct elements x and y such that (x, y) is nilpotent, then G is finite- by-nilpotent. 
The following result is an easy consequence of the latter which may be considered 
as an answer to a Erdos like question on non-nilpotent graphs. 

Theorem 4.1. (|20j) If G is a finitely generated solvable group, then Mg has no 
infinite clique if and only if the clique number of Ng is finite. 
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Let n G N. We say that a group G satisfies the condition (Af, n) whenever in 
every subset with n + 1 elements of G there exist distinct elements x, y such that 
(x, y) is a nilpotent group. This simply means that a group G satisfies the condition 
(A/", n) whenever every clique of *XIg has size at most n. Endimioni in [10] , proved 
that if n < 20, then every finite group satisfying the condition (Af,n) is solvable 
and moreover the alternating group A§ of degree 5 satisfies the condition (TV, 21). 
Also if n < 3, every finite group satisfying the condition (A/", n) is nilpotent. Now 
we can summarize the latter results in terms of non-nilpotcnt graph as following. 

Theorem 4.2. ( [10] ) A finite group G is nilpotent if and only if u>(Mg) < 3 and if 
u)(Ng) < 20, then G is solvable. 

It is asked in [2J Question 1] that: 

Question 4.3. Is there a non weakly nilpotent group G with uifflc) < 3? 

Some partial answers to this question are given in [2J Corollaire 2.2]. 

Tomkinson in [24], proved that if G is a finitely generated solvable group satis- 
fying the condition (Af,ri), then | \ < n n . This result gives a bound for the 
size of a finite solvable centerless group satisfying the condition (Af, n) . 

Theorem 4.4. (|24j) Let G be a finitely generated solvable group such that oj(A/g) 
is finite. Then the index of the hypercenter of G is finite and bounded above by 
(u:{Mg)Y {Ng)1 ■ 

In [5] more properties of groups in (Af, n) have been studied. These results can 
be stated in terms of non-nilpotent graph as follows. 

Theorem 4.5. Let G be a finite group without nontrivial normal abelian subgroups 
such that Lu(Afc) is finite. Then \G\ is bounded above by a function of lo(Mg)- 

Theorem 4.6. Let G be a finite non-solvable group. Then oj(A/g) = 21 if and only 

Theorem 4.7. Let G be a finite group. Then w(A/g) — 4 if and only if z *\g) — ^3- 

Here we give some general properties of non-nilpotent graphs. 

Proposition 4.8. Let G be a group which is not weakly nilpotent. Then diam(A/G) > 
2. Moreover, girth(A/c?) = 3. 

Proof. We show that there exists an element x £ G \ nil(G) such that x ^ x^ 1 . 
Suppose, for a contradiction, that a = a -1 for all a G G \ nil(G). In this case, we 
prove G = nil(G), which is a contradiction. Let x,y E G \ nil(G). If xy £ nil(G), 
then (x,y) is nilpotent, since (xy,y) = (x,y). If xy ^ nil(G), then xy = yx, since 
(xy) 2 = x 2 = y 2 = 1. It follows that (a, b) is nilpotent for all a,b 6 G. 
Therefore there exists an element x G G \ nil(G) such that x ^ x~ x . Since x G 
G\nil(G) , there exists an element y G G such that (x, y) — (x , y) is not nilpotent. 
Hence x — y — x" 1 is a path of length 2 and so diam(A/G) > 2. 
Suppose that x G G\nil(G). Thus there exists y G G\nil(G) such that x and y are 
adjacent. It follows that x — xy — y is a cycle in Afa- That is, girth(A/c) = 3. □ 

Proposition 4.9. There is no finite non-nilpotent group G with a non-nilpotent 
proper subgroup H < G such that OTg — ■ 
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Proof. Suppose, for contradiction, that there is a finite non-nilpotent group G with 
a non-nilpotent proper subgroup H satisfying 91g — Olff. Then \G\ — \nil(G)\ = 
\H\ - \nil{H)\ and so by Proposition O we have \G\ - \Z k (G)\ = \H\ - \Zi(H)\, 
where k and I are positive integers. Clearly, \H\ < \\G\ and |Zfc(G)| < \\G\. It 
follows that \G\ = \H\ - \Z t (H)\ + \Z k (G)\ < \G\, which is a contradiction. □ 

Proposition 4.10. There is no finite non-nilpotent group G with a normal sub- 
group N <G such that G/N is non-nilpotent and 91g — 91g . 

Proof. Suppose, for contradiction, that there is a finite non-nilpotent group G with 
a normal subgroup N < G such that G/N is non-nilpotent and 91g — 91g . Then 

\G\ - \nil{G)\ = |f | - \nil(§)\ and so by Proposition O we have |G| - \Z k (G)\ = 
f | — |Z;(f )|, where k and I are positive integers. Now by Case 1 of Lemma [2.21 
and Proposition ^. II 



G > Z^N 



It follows that 



\G\-\Z k (G)\ = &- \Z l{ g)\ <' G - MG)N<_ t G t , Z,(G) 



N 1 v iV yl - 'TV' N N 'NnZi(G) r 

Let |G| = m, \N\ = n, \Z k {G)\ = n, \N R Z t (G)\ = t, \Z t (G)\ = r 2 and a = a l t 
follows that m — n < ^ — ^ and so ^ < i— f- < j— 4- < 5, since n > 6. Hence 
' is a nilpotent group and so G is a nilpotent group, a contradiction. □ 



Z k (G) 



Proposition 4.11. Let G be a non-nilpotent finite solvable group. Then 

\E(m G )\ > ^|G| 2 , 

2p 

where p is the smallest prime number dividing the order of G. 
Proof. By Theorem 5 of [T2] , vq > , where Vq denotes the proportion of ordered 
pairs of G that generate a non-nilpotent subgroups. Therefore vo = 2 ' "^ G °|2 G ^ > 
and so |^(91g)| > ^|G| 2 . □ 

5. Connectedness of the non-nilpotent graphs 

In this section study the connectedness of 91g f° r a finite non-nilpotent group 
G. We prove that 91g is connected and dim(9TG) < 6. Moreover some results and 
examples suggest that dim(9lG) = 2 for any finite non-nilpotent group G. 

Theorem 5.1. Let G be a finite non-nilpotent group. Then 91g * s connected and 
its diameter is at most 6. In particular, every two vertices x and y with it(x) ^ 7r(y) 
are connected by a path of length at most 4. 

Proof. We first prove that if x and y are two distinct vertices of 9Tg which are of 
prime power orders, then there is a path connecting them of length at most 6. 
Suppose x and y are two p— and g-elements of G\nil(G), where p and q are two 
distinct prime numbers. We prove that there is a path connecting x and y of 
length at most 4. We may assume that x and y are not adjacent. This means 
that (x,y) is nilpotent and since gcd(|a;|, |y|) = 1, we have (x, y) = (xy). Now 
since x,y $ m/(G), there are elements x' and y' which are adjacent to x and y, 
respectively. Since (xy,x') = (x,y,x') and (x,x') is not nilpotent, it follows that 
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xy and x' are adjacent. By a similar argument, xy is also adjacent to y' and so 
x — x' — xy — y' — y is a path of length 4. 

Now suppose that x and y are two p-elements of G\nil(G) for some prime p. Since 
G is not nilpotent, there exists an g-element z G G\nil(G) such that p ^ q. Since 
x,y,z are not in nil(G), there are elements x',y',z' which are adjacent to x,y,z, 
respectively. By a similar proof 

x — x — xz — z — yz — y — y 

is a path of length 6 connecting x and y. 

Now let x and y be two arbitrary elements in G\nil(G). Then there are integers n 
and m such that x n ,y rn £ nil{G) and they are of prime power orders. Thus by the 

above, there is a path x n — x\ — ■ Xk — y m such that k < 3 if gcd(|cc™|, \y m \) = 1 

and k < 5 if gcd(|x ra |, \y m \) ^ 1. Since {x n ,xi) < (x,xi) and (y"\xk) < (y,xt), x 
is adjacent to x\ and y is adjacent to x^. Hence x — x\ — ■ ■ ■ — Xk — y is a path of 
length fc + 1 connecting x to y. Note that if it{x) ^ ""(y), then we may choose n 
and m such that gcd(|cc™|, \y m \) = 1 and so in this case x and y are connected with 
a path of length at most 4. 

□ 

Question 5.2. What is := max jdiam(9TG) G is a finite non-nilpotent group} ? 

By the theorem above < 6. 

Theorem 5.3. Let G be a finite group having no nontrivial normal abelian sub- 
group. Then diam(OTG) = 2. 

Proof. Since G has no nontrivial normal abelian subgroup, the largest normal solv- 
able subgroup S(G) of G is trivial. Since a nontrivial nilpotent group has nontrivial 
center, it follows from Lemma T2.ll that nil(G) = 1 . Now also by Theorem 6.4 of 
[15] . for every two nontrivial elements x and y in G there exists an element z in 
G such that (x, z) and (y, z) are not solvable subgroups and so are not nilpotent 
subgroups. This implies that the graph O^g is connected and diamfflc) — 2. □ 

Proposition 5.4. Let G be an n-group which is not weakly nilpotent. Then 9Tg is 
a connected graph and diam(OtG) = 2. 

Proof. Suppose, for a contradiction, that diam(G) > 2 (note that by Proposition 
14.81 diamffic) 7^ 1). Thus there exist two vertices x and y of such that 
d(x,y) 2, and so G — nilc(x) U nilc{y). Now since G is an n— group, it follows 
that either G = uUq(x) or nilc{y) — G. This gives a contradiction, as x and y are 
not in nil{G). □ 

6. Groups whose non-nilpotent graphs are planar 

A planar graph is a graph which can be drawn in the plane so that whose edges 
intersect only at end vertices. Note that every subgraph of a planar graph is also 
planar. 

Theorem 6.1. Let G be a finite non-nilpotent group. Then Vic is planar if and 
only if G = S3. 
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Proof. Since the Engel graph Eg is a subgraph of 9Tg (see [T] for the definition), the 
graph Eg is a planar graph. It follows from Theorem 3.1 of [I] that G is isomorphic 
to one of the following groups: 

S 3 , Dis or T = (x,y \ x 6 = x 3 y- 2 = x y x = 1). 

On the other hand by Lemmas 13.51 and 13 A\ the groups T and D\i are n-groups and 
nila{a) = Ca{a), for all a G T or £>i 2 . Therefore for these cases we have Tq — 
where Tq is the commuting graph of G (see [3] for the definition) . Now Proposition 
2.3 of [5] implies that Tq is planer if and only if G ~ S3, Dg or Q%. It follows that 
G = S3. The converse is clear. □ 

Corollary 6.2. Let G be a non-nilpotent group such that VIq = 91s 3 , then G = S3. 

Proof. It follows from Theorem 16. 1[ that the only non-nilpotent group whose non- 
nilpotent graph is planer, S3. Hence G = S3. □ 

7. Groups whose non-nilpotent graphs are regular 

In this section we give a characterization of finite nilpotent groups in terms of 
non-nilpotent graphs. In this section we prove that 

Theorem 7.1. A finite group G is nilpotent if and only if the set of vertex degrees 
of Mg has at most two elements. 

Note that the elements in nil(G) has degree (isolated vertices) in Mg\ so what 
we want to prove in this section is equivalent to this statement: There is no finite 
non-nilpotent group G such that is a regular graph. 

Throughout this section, suppose, for a contradiction, that G is a finite non- 
nilpotent group such that Olg is regular. Thus \nilG(x)\ — |mi(j(y)| for all 
G\nil(G). Note that, by Lemma l2~2"l the non-nilpotent graph of H — z ?{g) i s a ^ so 
regular and Z*(H) = nil(H) = 1. Therefore we may assume that nil(G) = 1 and 
\nilG(x)\ — \nilG(y)\ for any two nontrivial elements x, y of G. 

Lemma 7.2. For all nontrivial elements a G G, we have ni/c(a) = ri^c(a z ) for 
all 1 < i < \a\ — 1 In particular, (a,x) is not nilpotent if and only if {a l ,x) is not 
nilpotent for some i G {1, . . . , |a| — 1}. 

Proof. It is clear that uUg (a) C niZc(a J ). Thusn^c( a ) — nilG{a l ) since |m/o(a)| = 
ItmZg^)! for all i G {1, . . . , \a\ - 1}. □ 

Lemma 7.3. For all nontrivial elements a, b G G such that ab — ba we have the 
following: 

(1) If \a\ < \b\ then nilQiob) = nilG{b). 

(2) J/gcd(|a|, = 1, then nilaip) = nilcib) = nila{ob)- 

Proof. (1) Suppose that c G nilG(ab), it follows that (c, ab) is a nilpotent subgroup, 
so (c, (a6)l a l) = (c, 6' a ') is a nilpotent subgroup and so by Lemma [721 ( c i b) is a 
nilpotent subgroup, that is c G n«Zc(&). Thus nila(ab) C nilcib) and so nila(ab) = 
nilcib), since \nilG(ab)\ — \nilG(b)\. 

(2) It is enough to nilG(ab) C nil(a) H niloib). Let c G nilG(ab), so (c,ab) is a 
nilpotent subgroup. It follows that (c, (a6)' a ') = (c,6' a ') and (c, (afr)' 6 '} = (c,a' b ') 
are nilpotent subgroups and so two subgroups (c, b) and (c, a) are nilpotent, since 
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nilc{b) — nilG(b^ al ') and nilG(a) = nilG(a^). Thus c £ niloia) H nila{b), namely 
nilciab) C nila(o>) f~l nila{b). Therefore by Case (1) and regularity, nilciab) = 
nilcia) — nilcib). □ 

Lemma 7.4. // a is an element of G which is not of prime power order, then 
nilc(a) is a nilpotent Hall subgroup of G. 

Proof. By hypothesis, there are integers i and j such that \a l \ — p and \a J \ = q 
are distinct prime numbers. By Lemma l7.2[ nilQ{a l ) = nilc^a 3 ) — nilc{a). Let 2/1 
and 2/2 be two nontrivial p'-elements of nilcia 1 ). Thus [yi,a l ] = [2/2,0'] = 1 and 
so nil G {a l ) = nil G (yi) = nilGfyz). Therefore (2/11/2,2/2) = (2/1,2/2} is nilpotent. It 
follows that 2/12/2 £ nila{a) and so 

A = { y E nilc^a) \ y is an p'-element}, 

is a subgroup and as every pair of elements of A generates a nilpotent subgroup, 
A is a nilpotent p'-group. Now let Z\, z-i be two nontrivial p-elements of nilaiaP). 
Then by a similar argument nilc(a) — nilc(zi) — nilciz-i) and so 

B = { y € 72.^0(0)! y is an p — element} 

isp-subgroup. Also we have that nila (a) = nilcix) — nila {y) for any two nontrivial 
elements x € A and y G B. Hence (x,y) is nilpotent and so [^4,5] = 1- It is clear 
that nila(a) C AB and so it follows that nilcia) = AB is a nilpotent group. 
It remains to prove n^c(a) is a Hall subgroup of G. Let r be a prime number 
dividing \nilG{a)\. Then either r ^ p or r 7^ q. Assume without loss of generality 
that r ^ p and let a; be a nontrivial r-element of nilcia). Note that nilcix) = 
Hilda), by the first part of the proof. Now if Q is any Sylow r-subgroup of G 
containing x, then Q < uUq(x). It follows that mZc(a) is a Hall subgroup of 
G. □ 



Lemma 7.5. Every element of G has prime power order. 

Proof. Suppose, for a contradiction, that there is an element of G which is not of 
prime power order. Then by Lemma l7.4i nilo{a) is a nilpotent Hall subgroup of G. 
Since G is assumed to be non-nilpotent, there is a prime number p dividing \G\ and 
p j \nilG{a)\. Let x be nontrivial p-element of G. Then by Lemma 12. 3[ p divides 
|niZ(3(a;)| = \nilG(a)\, a contradiction. This completes the proof. □ 

Lemma 7.6. Let x be a nontrivial p- element of G for some prime p. Then 

nilcix) = \ \{P \ P is a Sylow p-subgroup containing x}. 

In particular, if a Sylow p-subgroup of G is abelian, then nilc(x) — Cg{x) and so 
nilc(x) is a p-subgroup. 

Proof. Suppose that y G nilc{x). Thus (x,y) is nilpotent and so it follows from 
Lemma 17.51 that (x, y) is a p-subgroup. Hence (x, y) is contained in a Sylow p- 
subgroup of G. This completes the proof. □ 

Lemma 7.7. G is neither solvable nor having a normal Sylow p-subgroup for some 
prime p dividing \G\. 



12 



ALIREZA ABDOLLAHI AND MOHAMMAD ZARRIN 



Proof. Suppose, for a contradiction, that G is either solvable or having a normal 
Sylow p-subgroup for some prime p dividing \G\. It follows from Lemma 17.51 and 
[171 Theorem 1] that G contains either a cyclic Sylow g-subgroup for some prime 
q dividing \G\ or a normal Sylow p-subgroup for some prime p dividing \G\. Thus, 
by Lemma 17.61 in any case there exists an element x € G such that nilcix) is (a 
subgroup) of prime power order. Since G is not nilpotent, \G\ is dividing by at least 
two primes. Now Lemma 12.31 gives a contradiction. □ 

Completion of the Proof of Theorem [7711 By Lemma [7771 G is neither solvable 
nor having a normal Sylow p-subgroup for some prime p dividing |G|. Now using 
the classification of finite groups in which every element has prime power order (see 
[6] or O Main Theorem] ) , it is easy to see that G has a cyclic Sylow g-subgroup 
for some prime q dividing \G\ (one must only check the cases (4) and (5) of [HI Main 
Theorem] for this; the other cases are ruled out by Lemma [7.7p . Now Lemmas 
17.61 and 12.31 complete the proof. 

The converse is clear. □ 
Acknowledgements. The research of the first author was partially supported by 
the Center of Excellence for Mathematics, University of Isfahan and he gratefully 
acknowledges the financial support of University of Isfahan for the sabbatical leave 
studies in University of Bath, UK and ICTP, Trieste, Italy. 

References 

[1] A. Abdollahi, Engle graph associated to a group, J. Algbera, 318 (2007), no. 2, 680-691. 
[2] A. Abdollahi, Groupes satisfaisant une condition de nilpotence, Math. Rep. (Bucur.) 8(58) 
(2006), no. 4, 385-389. 

[3] A. Abdollahi, A. Akbari and H.R. Maimani, Non-commuting graph of a group, J. Algbera, 

298 (2006), no. 2, 468-492. 
[4] A. Abdollahi and A. Mohammadi Hassanabadi, Non-cyclic graph of a group, Comm. Algebra 

35 (2007), no. 7, 2057-2081. 
[5] A. Abdollahi and A. Mohammadi Hassanabadi, Groups with a certain number of elements 

pairwise generating a non-nilpotent subgroup, Bull. Iranian Math. Soc. 30 (2004), no. 2, 1-20. 
[6] W. Bannuscher and G. Tiedt, On a theorem of Deaconescu, Rostock. Math. Kolloq. No. 47 

(1994), 23-26. 

[7] C.J.B. Brookes, Engel elements of soluble groups, Bull. London Math. Soc. 18 (1986), no. 1, 
7-10. 

[8] C. Casolo, Finite groups in which subnormalizers are subgroups, Rend. Sem. Mat. Univ. 

Padova 82 (1989), 25-83. 
[9] A. Delgado and Yu-Fen Wu, On locally finite groups in which every element has prime power 

order, Illinois J. Math. 46 (2002), no. 3, 885-891. 
[10] G. Endimioni, Groupes finis satisfaisant la condition (J\f,n), C. R. Acad. Sci. Paris Ser. I 

Math. 319 (1994), no. 12, 1245-1247. 
[11] G. Endimioni, Groups covered by finitely many nilpotent subgroups, Bull. Austral. Math. 

Soc. 50 (1994), no. 3, 459464 
[12] J. Fulman, M. Galloy, G. Sherman and J. Vanderkam, Counting nilpotent pairs in finite 

groups, Ars Combin. 54 (2000), 161-178. 
[13] The GAP Group, GAP-Groups, Algorithms, and Programming, Version 4.4; 2005, 

(http:/ /www. gap-system, org ). 
[14] D. Gorenstein, Finite groups, Harper & Row, New York, 1968. 

[15] R. Guralnick, B. Kunyavskii, E. Plotkin and A. Shalev, Thompson-like charactrizations of 

the solvable radical, J. Algebra, 300 (2006), no. 1, 363-375. 
[16] H. Heineken, On E-groups in the sense of Peng, Glasgow Math. J. 31 (1989), no. 2, 231-242. 
[17] G. Higman, Finite groups in which every element has prime power order, J. London Math. 

Soc. 32 (1957), 335-342. 
[18] B. Huppert, Endliche Gruppen I, Springer, Berlin, 1967. 



NON-NILPOTENT GRAPH OF A GROUP 



13 



[19] B. Huppert and N. Blackburn, Finite groups, III, Springer, Berlin, 1982. 

[20] J. C. Lennox and J. Wiegold, Extensions of a problem of Paul Erdos on groups, J. Austral. 

Math. Soc. 31 (1981), no. 4, 459-463. 
[21] B. H. Neumann, A problem of Paul Erdos on groups, J. Aust. Math. Soc. Scr. A 21 (1976), 

no. 4, 467-472. 

[22] T.A. Peng, Finite soluble groups with an Engel condition, J. Algebra 11 (1969), 319-330. 
[23] D. J. S. Robinson, A course in the theory of groups, Springer, New York, 1982. 
[24] M.J. Tomkinson, Hypercentre-by-finite groups, Publ. Math. Debrecen 40 (1992), no. 3-4, 
313-321. 

Department of Mathematics, University of Isfahan, Isfahan 81746-73441, Iran 
E-mail address: a. abdollahiamath.ui . ac . ir , zarrinl380Syahoo.com 



